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Abstract 

Quasianalytic contractions form the crucial class in the quest for proper 
invariant and hyperinvariant subspaces for asymptotically non-vanishing 
Hilbert space contractions. The property of quasianalycity relies on the 
concepts of unitary asymptote and //“-functional calculus. These objects 
can be naturally defined in the setting of polynomially bounded operators 
too, which makes possible to extend the study of quasianalycity from 
contractions to this larger class. Carrying out this program we pose also 
several interesting questions. 
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1 Introduction 

Let fit be a complex Hilbert space, and let CfiH) stand for the C*-algebra of 
all bounded linear operators acting on %. Given any T £ £(%), the complete 
lattice Lat T consists of those (closed) subspaces A4 of ft, which are invariant 
for T : T A4 C A4. A subspace A4 is hyperinvariant for T, if it is invariant for 
every operator C £ C(fi), commuting with T : CT = TC. The complete lattice 
of all hyperinvariant subspaces of T is denoted by Hlat T. The most challenging 
open questions in operator theory are arguably the Invariant Subspace Problem 
(ISP) and the Hyperinvariant Subspace Problem (HSP). The first question asks 
whether LatT is non-trivial (i.e., different from {{0},%}), for every operator 
T £ CfiH), provided dim ft >2-, while the second question asks whether HlatT 
is non-trivial, whenever T is not a scalar multiple of the identity operator I. 
For a thorough discussion of these problems see the monographs [RR] and [CP], 
and the more recent papers [AM] and [GR]. 

Of course, dividing T by its norm, we may assume that T is a contraction: 
||T|| < 1. An opposite, local estimate is made, when we assume that the contrac¬ 
tion T is asymptotically non-vanishing , that is lim„ ||T n /i|| > 0 holds for some 
h £ ft. Suprisingly enough, (ISP) and (HSP) are open under these assumptions 
too. A useful extra tool available in this situation is the unitary asymptote 
of T. Relying on this and on the //“-functional calculus, two spectral invari¬ 
ants can be introduced for T. The first one is the residual set uj(T), and the 
second one is the quasianalytic spectral set 7r(T) of T. The contraction T is 
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called quasianalytic, if these measurable subsets of the unit circle T coincide: 
ui(T) = 7 r(T). Quasianalytic contractions were investigated in the papers [K5], 
[K8], [K9], [KT], [KSz2] and [KSz3]. A central theorem proved is that (HSP) 
can be reduced to this class in the asymptotically non-vanishing case. 

The aim of this paper is to extend these investigations from contractions to 
polynomially bounded operators. It will turn out that this larger class is the 
natural setting for the study of quasianalycity. 

Our work is organized in the following way. 

In Section 2 we introduce unitary asymptotes for an arbitrary operator T £ 
C{1~L) in a categorical sense, as it has been done for contractions in [BK] and 
in Chapter IX of [NFBK], The induced generalized Toeplitz operators, and the 
connection of the associated symbolic calculus with the commutant mapping 
are discussed. Existence of unitary asymptotes in the power bounded case is 
shown by using Banach limits. This method originates in Sz.-Nagy’s pioneering 
work [N], and has been extended to many situations; see, e.g., [Kl], [K2], [K3], 
[K4], [K6], [KL], [KM] and [Pr]. A new characterization is given in terms of 
nor m- conditions. 

It is well-known that every contraction T can be decomposed into the or¬ 
thogonal sum T = T a © U s of an absolutely continuous contraction T a and a 
singular unitary operator U s . Mlak showed that analogous Lebesgue-type de¬ 
compositions can be given for polynomially bounded operators, or equivalently, 
for bounded representations of the disk algebra A(T). Actually, Mlak considered 
representations of more general function algebras; see [Ml], [M2], [M3] and [M4]. 
Using Mlak’s elementary measures, in Section 3 we give a detailed, streamlined 
discussion of the Lebesgue decomposition in the particular case M(T), what is of 
the main interest for us. Our purpose here (and partly in the next two sections) 
is to make Mlak’s important results more accessible for a wide range of readers. 

In Section 4 we focus on intertwining relations. It is verified that absolute 
continuity and singularity are preserved under quasisimilarity. A transparent 
proof is given for the important known fact that every singular polynomially 
bounded operator is similar to its unitary asymptote. Furthermore, the problem 
of similarity to contractions is discussed. 

Sz.-Nagy and Foias introduced the effective functional calculus, working with 
functions in H°°, for absolutely continuous contractions; see Chapter III in 
[NFBK], Section 5 is devoted to the study of properties and possible range 
of this //“-functional calculus. A simplified proof is given for Mlak’s result, 
stating that exactly the absolutely continuous polynomially bounded operators 
admit //“-functional calculae. 

In Section 6 we turn to quasianalytic operators. The sets ui(T) and n(T) 
are introduced in a uniform way, relying on the local residual sets ui(T,x) (x £ 
H). The quasianalytic spectral set 7r(T) is characterized also in terms of non¬ 
monotone sequences. The central hyperinvariant subspace theorem is proved. 
Furthermore, the effects of intertwining relations, the asymptotic behaviour, 
orthogonal sums and restrictions are studied. 

For the theory of Hardy spaces H p , see [Ho]. In connection with the theory of 
contractions we refer to [NFBK], Given operators Ti £ C(Hi) and Ti £ C(V. 2 ), 
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the set of intertwining transformations is defined by 

2(Ti,T 2 ) := {C £ C{HuH 2 ) ■ CT X = T 2 C} . 

The commutant of the operator T £ £(77) is {T}' = X(T,T), while the bicom- 
mutant (or double commutant) {T}" of T consists of those operators B £ £(77) 
which commute with every operator C in {T}'. Finally, N, Z, Z+, R, R + , C de¬ 
note the set of positive integers, integers, non-negative integers, real numbers, 
non-negative real numbers and complex numbers, respectively. 


2 Unitary asymptotes 

Let 77 be a complex Hilbert space, and let us consider an operator T £ £(77). 
We say that (X, U) is a unitary intertwining pair for T, if U is a unitary op¬ 
erator acting on a (complex) Hilbert space K., and X £ X{T,U). The pair is 
minimal , if V%L 1 U~ n XXL = 1C. The unitary intertwining pair (X,U) is called 
a unitary asymptote of T, if for any other unitary intertwining pair (X', U') of 
T, there exists a unique transformation Z £ X(U, U') such that X' = ZX. The 
uniqueness of Z implies that a unitary asymptote (X, U) is necessarily minimal. 

Let us assume that (Xi,Z7i) and (X 2 ,t/ 2 ) are unitary asymptotes of T. 
Then there exist Z\ £ X{U\,U 2 ) and Z 2 £ X(U 2 , U\) such that Z\X\ = X 2 and 
Z 2 X 2 = Xi. Therefore, I-X\ = Z 2 X 2 = ( Z 2 Zf)X\ , whence 7 = Z 2 Z\ follows by 
the definition of the unitary asymptote. The equation Z\Z 2 = I can be derived 
similarly. Thus, the unitary intertwining pairs (Xi,Ui) and (X 2 ,C/ 2 ) are equiv¬ 
alent, which means the existence of an invertible transformation Z £ X(U\, U 2 ) 
satisfying X 2 = ZX i. Then the unitary operators U\ and U 2 are necessarily 
unitarily equivalent: U\ cs U 2 . Furthermore, if the unitary intertwining pair 
(X 2 ,C/ 2 ) is equivalent to a unitary asymptote (Xi,Ui), then (X 2 ,U 2 ) is also a 
unitary asymptote. 

For any unitary intertwining pair (X,U) of T we have ||X/i|| = ||t/"X/i|| = 
\\XT n h\\ < ||X|| • \\T n h\\ (n £ N), and so 

\\Xh\\ < ||X|| ■ liminf \\T n h\\ for all h £ H. 

n—> oo 

A lower estimate for ||Xft,|| yields universality. 

Proposition 1 Let (X, U) be a minimal unitary intertwining pair for T. If 
there exists k £ (0, oo) such that 

k ■ liminf \\T n h\\ < \\Xh\\ 

n—^oo 

holds for all h £ XL, then (X, U) is a unitary asymptote of T. 

Proof. Let (X', U’) be a unitary intertwining pair for T. For every h £ XL we 
have 

||X'/i|| < IIX'H - liminf \\T n h\\ < ||X'|| - • \\Xh\\. 

n—x oo 
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Hence there is a unique transformation Y + £ C(/C + ,IC) such that Y + Xh = 
X'h ( h £ LL), where K+ = (XTL)~. The relations 

Y + UXh = Y+XTh = X'Th = U'X'h = U'Y + Xh (h £ H) 

show that Y + £ T(U+ , I/'), where U+ = U |/C + . It is immediate that the equa¬ 
tions 

YU~ n k := U'~ n Y + k (k £ fC + ,n £ N) 

define a norm-preserving extension Y £ I{U, U r ) of Y + . It is clear that YX — 
X 1 , and that Y is uniquely determined by these properties. 

□ 

Question 2 Does Klimm{ n ^ yoo \\T n h\\ < \\Xh\\ (h £ H) hold with a k £ (0, oo), 
whenever (X, [/) is a unitary asymptote ofT? 

Let (X, [/) be any unitary intertwining pair for T. Then T*(X*X)T = 
X*U*UX = X*X. Thus X*X belongs to the set 

T(T) := {B £ C{U) : T*BT = B} 

of T-Toeplitz operators. It is clear that T(T) is a selfadjoint linear manifold, 
which is closed in the weak operator topology. Taking T S (T) = {B £ T(T) : 
B* = B}. we have T{T) = T S (T) + iT s (T). The set T+(T) = {B £ T S (T) : 
B > 0} is of particular interest. We recall that A £ 7+(T) is universal ( weakly 
universal) in T(T), if for every B £ T S {T) ( B £ 7+(T), respectively) there 
exists £ R+ such that 

~/3A < B < (3A. 

It is obvious that T{U) = {U} 1 . Furthermore, for any F £ {U}' we have 
T*(X*FX)T = X*U*FUX = X*U*UFX = X*FX, that is X*FX £ T(T). 
The positive, bounded, ^-linear mapping 

: {U}' -> T(T), F ^ X*FX 

is the symbolic calculus for T(T), associated with (X, XJ). It is easy to see that 
T v is injective, when (X, U) is minimal. We say that Tx is universal , if its 
range coincides with T(T). The following theorem, proved in [K6] in a slightly 
different form, establishes connections among these concepts. 

Theorem 3 Let (X, U) be a minimal unitary intertwining pair for T. Then 

(i) (ii) <= (iii) (iv), where 

(i) (X, U) is a unitary asymptote of T; 

(ii) A = X*X £ T+(T) is weakly universal in T(T); 

(iii) A = X*X £ T+(T) is universal in T(T); 

(iv) the symbolic calculus is universal. 
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We note that if A = X*X is universal in T(T), then : {U}' —> T(T) is 
invertible, and so 

-0o\\B\\A<B<0o\\B\\A 

holds for every B £T S (T), where /3o = ||ft 

It may happen that there is not a universal operator in T(T); we refer to 
Example 5 in [K6]. Moreover, if T(T) contains a non-zero operator B , then the 
relations 0 < ||f?|| = \\T* n BT n \\ < ||.B|| ||T”|| 2 (n £ N) show that the spectral 
radius r(T) > 1. 

Let us assume that (X, U) is a unitary asymptote of T. Then, for every 
C £ {T}',XC £ I{T,U) holds, and so there exists a unique D £ {U}' such 
that XC = DX. It is easy to verify that the commutant mapping 

lx : {T}' {Uy, C^D 

is a (unital) algebra-homomorphism. Boundedness of jx follows from its rela¬ 
tion to 'lx- 

Proposition 4 Let (X, U) be a unitary asymptote of T, and let us assume 
that A = X*X is universal in T{T). Then (fx °7 x)C = AC holds for every 
C£{T}’, and so || 7 x|| < \\A\\ • ||'L* 1 ||. 

The simple proof is left to the reader. 

Existence of unitary asymptotes can be verified in the class of power bounded 
operators. We recall that T £ C(TL) is a power bounded operator , if 

Mt ’■= sup{||T"|| : n £ Z + } < oo. 

In that case r(T) < 1 is evidently true. If r(T) < 1, then lim^^oo ||T"|| = 0, 
and so, for any unitary intertwining pair (X, U) of T, the equations XT” = 
U n X (n £ N) imply that X = 0. Thus U acts on the zero space, provided 
(X, U) is minimal. Therefore r(T) = 1, whenever the power bounded operator 
T has a non-trivial unitary asymptote. 

Taking a Banach limit L: £°° —» C, there exists a unique operator Al £ C{TL) 
such that 

(Alx,p) = L- lim (T* n T n x,y) for all x,y £ TL. 

n—> oo 

It is evident that Al £ T+(T), and Al < Mf ■ I. Setting any B £ T S (T), the 
relations ( Bx,x ) = (BT n x,T n x) and 

-\\B\\(T n x,T n x) < {BT n x,T n x) < \\B\\(T n x,T n x) (x £H,n£ N) 

show that 

-\\b\\a l <b<\\b\\a l . 

Therefore, Al is universal in T(T). 

Since T*AifT = Al implies ||^4^ 2 T/i|| = ||7l^ 2 /i|| ( h £ TL), there exists 

1 /2 

a unique isometry Vj J on the space IC+ t L = (A^lT)- satisying the condition 
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VjA\[' 2 = A 1 / 2 T. Let Ul £ £(/Cl) be the minimal unitary extension of Vl, and 
let Xl £ C{'H,1Cl) be defined by X^h := A ^ 2 h (h £ H). Clearly, ( Xl,Ul) is a 
minimal unitary intertwining pair for T. Taking into account that X^Xl = Al 
is universal in T(T), we infer by Theorem 3 that (Xl, Ul) is a unitary asymptote 
ofT. 

We recall that the possible values of Banach limits on a real sequence £ £ t°° 
comprise a closed interval determined by the numbers 


Bffff) = S_- lim £(n) := sup < lirninf - £(n,- + k) : m ,... ,n r £ N,r £ N 

n —>oo I k—>oo T z —' 

and 


j=l 


B(0 = B- lim f(n) := inf < limsup - £(?r, + &) : n i, ■ ■ ■, n r £ N, r £ N ^ . 

n ^°° I k->oo r ^ 1 

For any h £ TL, we have 

\\X L h\\ 2 = (A L h,h) = L- lim \\T n h\\ 2 , 

n—> oo 


whence 


B- lim \\T n h\\ 2 < \\X L h\\ 2 < B- lim \\T n h\\ 2 . 

n—>oo n —>00 


These relations immediately yield that 

lirninf \\T n h\\ < \\X L h\\ < limsup \\T n h\\ (h £ H). 

n— ^°° n^-00 

We obtain the following characterization. 


Theorem 5 Let ( X , U) be a minimal unitary intertwining pair for the power 
bounded operator T £ C(TL). Then (X,U) is a unitary asymptote of T if and 
only if there exists a k £ (0, 00 ) such that 

k • lirninf \\T n h\\ < \\Xh\\ for all h£%. 

n—>• 00 

Proof. If ( X , U) is a unitary asymptote of T, then there exists an invertible 
Z £ T(U, Ul) such that Xl = ZX, with a Banach limit L. Thus, for every 
h £%, we have 

\\Xh\\ > WZW^WZXhW = WZW^WXlHW > ll^- 1 lirninf \\T n h\\. 

n—> 00 

The reverse implication follows by Proposition 1. 

□ 

As immediate consequences, we may derive the following statements. 


Corollary 6 Let (Xj , Uj) be a unitary asymptote of the power bounded operator 
Tj, for j = 1, 2. Then (X\ © X%, U\ © U 2 ) is a unitary asymptote of T\ © T^. 
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Corollary 7 If ( X , U) is a unitary asymptote of the power bounded operator 
T £ C[TL) and A4 £ Lat T, then (X\A4,U\M) is a unitary asymptote of the 
restriction T\M, where M := \/ n ^nU~ n XM. 

If T £ C(fH) is a contraction, then the decreasing sequence {||T'”/j||}J)T 1 is 
convergent, for every h £ TL. Suprisingly, these sequences are convergent also in 
the case, when T is a p-contraction; see [E] and [CC]. 

If T £ C(U) is an arbitrary power bounded operator, then for every h £ TL 
we have 

lira sup \\T n h\\ < M t liminf \\T n h\\. 

n—>oo n^-oo 

In particular, inf{||T n /i|| : n £ N} = 0 implies lim^oo ||T"/i|| = 0. In view of 
the previous observations we may infer the following statement. 

Proposition 8 If ( X , U) is a unitary asymptote of the power bounded operator 
T £ C{%), then the nullspace of X coincides with the hyperinvariant subspace 
of those vectors, which are stable for T: 

ker X = H 0 {T) ={h£U : lim \\T n h\\ = o) . 

I n—>• oo J 

We conclude this section with norm-estimates concerning the symbolic cal¬ 
culus and the commutant mapping. 

Proposition 9 Setting a Banach limit L, let (Xl,Ul) be the corresponding 
unitary asymptote of the power bounded operator T £ C(fh). Then 

(i) 11*11 < ||*x i (* , )|| < M*\\F\\ for all F £ {U L }'; 

(ii) Mt 2 \\A l C\\ < \\jx L (C)\\ < \\A l C\\ for all C £ {T}'. 

Proof. For (i) see the proof of Theorem 3 in [K6]. Statement (ii) is a conse¬ 
quence of (i) and Proposition 4. 

□ 


3 Lebesgue decomposition 

Let V(T) denote the algebra of analytic polynomials, restricted to the unit circle 
T = {£ £ C : |e| = 1}. C(T) is the abelian Banach algebra of all continuous 
functions defined on T. The disk algebra _4.(T) is the norm-closure of 'P(T) in 
C'(T). We recall that the dual of C(T) can be identified with the Banach space 
M( T) of all complex Borel measures on T (which are automatically regular). 
Namely, by the Riesz Representation Theorem, the mapping 

T : M(T) -s- C(T) # , where W(/) = [ / d T, 

J T 

is a Banach space isomorophism. 
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We say that /x, v £ M(T) are analytically equivalent measures, in notation: 

CL • p 

y ~ i/, it 


w dy 


u du 


for all u £ .4.(1). 


It is clear that this is an equivalence relation. Furthermore, by a well-known 
theorem of F. & M. Riesz, y ~ v holds if and only if v = y + h dm with 
a function h £ Hi (see, e.g., [Ho]). Here and in the sequel m denotes the 
normalized Lebesgue measure on 1, and Hi = {/ £ H 1 : f T f dm = 0}. We 
note that if the measures y, v are singular (with respect to to), then y ~ v holds 
exactly when y = v. 

For any T £ C(fH), the polynomial calculus 




is the uniquely determined (unital) algebra-homomorphism, which transforms 
the identical function %(£) = £ (( el) into T. We say that T is a polyno¬ 
mially bounded operator, if $t,o is norm-continuous. In that case $ 7,0 can be 
continuously extended to a bounded algebra-homomorphism 


$r, 1 : A(T)^C(H), u u(T). 

Our aim in this section is to decompose T into the direct sum of an abso¬ 
lutely continuous component and a singular component. We shall follow Mlak’s 
method of using elementary measures. 

More generally, we consider and arbitrary bounded, linear transformation 
<F: .4(1) —> C(fH). For any x,y £ H, ip XtV (C) = (Cx,y) defines a bounded 
linear functional on £(%). By the Hahn-Banacli Theorem ip x , y o $ can be 
extended to a bounded linear functional on C(T), even in a norm-preserving 
way. The set of elementary measures of $ at x,y is defined by 

M($,x,y) := {n £ M( 1) : = ip XtV o $} . 

Therefore, p £ M{<&,x,y) precisely when 


($(u)x,y) = udy holds for all u £ 4,(1). 

J T 

We say that $ is absolutely continuous (a.c.), if for every x,y £ H the measures 
in M($>,x,y) are a.c. (with respect to to). On the other hand, $ is called 
singular, if for every x,y £ H the set M(&,x,y) contains a measure, which is 
singular (with respect to m). 

It is well-known that every measure y £ M(1) can be uniquely decomposed 
into the sum y = y a + y s , where y a £ M(1) is a.c. and y s £ M(T) is singular. 
By the next proposition such Lebesgue-type decomposition is valid also for $. 

Proposition 10 If .4(1) —> C(fH) is a bounded linear transformation, then 



(i) there exists a unique decomposition <F = <F a + <f> s , where the bounded 
linear mapping $°: *4(T) —>• CifH) is a.c., and the bounded linear mapping 
$ s : „4(T) —> C{%) is singular; 

(ii) ||<I>“|| < pH and |p s || < p||; 

(iii) for every x,y £ H, we have M($ a ,x,y) = {p a : p £ M($,x,y)} and 
M (<f> s , x, y) = {p s + hdm : p £ M(<f>, x, y),h £ Hq }. 

Proof. For every x,y £ ti, the singular component of the measures in M (<f>, x , y) 
is uniquely determined, denoted by p% x y - Taking a measure p £ M(f&,x,y) 
with minimal norm, we obtain 

11/4,*, ,11 <11/^11 = 11^0 $|| <p||-IMI- llvll. 


The mapping 

u! 4 : H x Tt — > M (T), (x,y) i—>• p% xy 

is sesquilinear (linear in x, and conjugate linear in y). For example, let us 
check the linearity in x. Setting xi,x%,y £ H, c £ C, let us choose pj £ 
M(&,Xj,yj) (j = 1,2) and p £ M($,cx i + x 2 ,y). Then, for every u £ »4(T), 
we have 


udp = ($(u)(cxi+x 2 ),y)=c{®(u)xi,y) + ($(u)x 2 ,y) 

J T 

= c udpi+ / udp 2 = / ud(cpi+p 2 ), 

J T J T J T 

whence p ~ cpi + p 2 , and so p s = (cpi + p 2 ) s = cp\ + follows. 

For any g £ C(T), let us consider the bounded linear functional 


A g : M(T) —>• C, p H > / g dp, 

J T 

with |||| = ||(/||. The bounded sesquilinear functional 
w%, g := A g o w% : H x % —>• C 

satisfies the condition ||to| || < p|| plH < ||g||p||. There exists a unique 
operator <F s (g) £ C{fH) such that 

( &{g)x,y ) = w%, g {x,y) = J g dp% >XtV 

holds for all x,y £ H; furthermore p s (g)|| = ||tu| )9 || < p|| • ||g||. Linearity of 
the integral in g yields that the mapping $ s : C(T) —► C(7~L) is linear; we obtain 
also that p s || < p||. Then 

:= $ S P(T): A( T) -»• C(H) 
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is also a bounded linear transformation, and ||$ s || < ||$||. Given any x,y £ 'H, 
for every u £ „4(T), we have 

(<I> s (u)x,y) = / udy | (X)3/ ; 

Jt 

hence ,x,y) = + hdm : h £ H q}, and so <F S is singular. 

Let us consider the bounded linear transformation <f>° := $ — <f> s . Given any 
x,y £ H and setting / 1 £ M($,x,y), for every u £ -4(T), we have 


($ a {u)x,y) = {$(u)x,y) - ($ s (u)x,y) = / udy- / udy 3 = / ud{y-y s ). 

J T JT JT 

Thus /x“ = /x — y s £ M($ a ,x,y), and so $° is a.c.. Choosing y so that ||/z|| = 
\Wx,y 0 $||, we obtain 

| {$ a (u)x,y) | < ||u|| • ||/z a || < ||u|| • ||/z|| < ||u||||$||||a;||||y||. 

Therefore ||$ a (u)|| < ||$|| • ||w|| (u £ .4.(1)), and so ||$ a || < ||$||. 

Finally, we turn to the uniqueness of the decomposition. Suppose that $ = 
= $2 + $2 are Lebesgue decompositions. Then $ = — $2 = — $f 

is simultaneously a.c. and singular. Thus, the elementary measures of $ are 
analytically equivalent to 0, and so $ = 0. 

□ 


Remark 11 We note that these considerations can be carried out in the Banach 
space setting also, with some restrictions. Let X be a complex Banach space, 
and let $: ,4(T) —» C(X) be a bounded linear transformation. For any x £ X 
and y £ X #, the measure y £ M( T) belongs to M($,x,y) if 


/ udy = [$(u)x,y] for all u £ .4(T). 

J T 

For any g £ C'(T), we can define the bounded bilinear functional 

w^g : X x X* C 

as before. Then a necessary and sufficient condition for the existence of an 
operator $ s (g) £ C{X) satisfying 


$ s (g)x,y 


w %, g (x,y), for all x £ X, y £ X#, 


is that g {x,y) be weak-* continuous in y. 

Now we turn to bounded representations of „4(T). We recall that a map¬ 
ping <f>: „4(T) —> C{T~L) is called a representation, if it is a unital algebra- 
homomorphism. Norm-continuous representations arise as functional calculae 
for polynomially bounded operators: $ = $t,i where T = $(x). 
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Uniqueness of the Lebesgue decomposition of measures implies transforma¬ 
tion rules for elementary measures. Given any p, v £ M( T), if p ~ v then 
v = p + hdm with some h £ Hq, and so p a ~ v a and p s = v s . Furthermore, 
for any v £ .4(T), the equation v dp = v dp a + v dp s yields that (v dp) a = v dp a 
and (v dp) s = vdp s . 

Lemma 12 Let <f>: ,4.(T) — > C(H) be a bounded representation. Setting x,y £ 
H andv £ -4(T), letx v := Q(v)x andy* := &(v)*y. If p X:V £ M($,x, y),p Xv , y € 
M($,x v ,y) and p x , y * £ M($,x,y*), then 


x v y 


V dpi 


x,y 


V S x v ,y = Vdp. 


x,yi 




V dp 


x,yi 


= v d P 


x,y’ 


Proof. For every u G A(T), we have 


/ uvdp Xt y = ($(uv)x,y) = ($(u)$(v)x,y) = ($(u)x v ,y) = / udp XvtV , 

J T J T 

whence p Xv ,y ~ vdp X}V , and so p% v V ~ vdp x y and p XvtV = vdp% y follow. The 
other two relations can be derived similarly from the equation (<f>(uv)x,y) = 
($(u)x, y*). 

□ 

We say that the polynomially bounded operator T £ C(7i) is absolutely 
continuous (singular) if its functional calculus $t,i is an absolutely continuous 
(singular, respectively) representation. 

Theorem 13 Let T £ £(71) be a polynomially bounded operator, and let us 
consider the Lebesgue decomposition <F = <f>° + of the bounded representation 
$ = $ T>1 ; yt(T) C(H). Then 

(i) <!>“ and are also bounded representations; 

(ii) P a = <F a (l) and P s = <f> s (l) are complementary projections belonging to 
the bicommutant {T}"; 

(iii) H = r H a + H s , where TL a = P a Tl and TL S = P s Tl are hyperinvariant 
subspaces of T; 

(iv) T a = T\TL a is an a.c. and T s = T\Ti s is a singular polynomially bounded 
operator; the Lebesgue decomposition T = T a + T S ofT is unique; further¬ 
more, <b a (it) = $T“, 1 (w)+0 and $ s (tt) = 0 + < f>T s ,i(w) for all u £ ^4(T). 

Proof, (i): Given x,y £ H, set p x>y £ M($,x,y). Then pf y £ M($ a ,x,y) 
and v pf y ~ P Xv} y by Proposition 10 and Lemma 12. Hence, for u, v £ „4(T), 
we have 

( <& a (uv)x,y) = [ uvdp a = [ udp Xv y = (<f> a {u)x v ,y) = (<F a (u)<F(u)£, y). 

J T J T 
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Consequently, 


^(ud) = $“(u)$(i)) for all u,v £ A(T). 


Similarly, 

($ a (uv)x,y) 

and so 

Then 



(<f> a (v)x,y*) = (<f> a (v)x : $(u)*y) = ($(u)$ a (v)x,y), 
$ a (uv) = $(«)$“(». 


<S> a (u)<S> a (v) + <f> a (u)<S> s (v) = <S> a (u)<S>(v)=<S> a (uv) = <S>(u)<S> a (v) 

= <f> a (u)<I> a (v) + <f> s {u)<S> a (v), 

whence 

$ a (u)$» = $ s (w)$ a (n) for all u,v £ A{ T). 

Fix v £ .4.(1), and set x,y £ H. Let x\ := <F s (i>);r and Xi := $ a (t>)a:. Then, for 
every u £ „4(T), we have 


[ udnl uy = {<S> a (u)x u y) = {§ a {u)$ s {v)x,y) 

J T 

= {<$> s {u)<5> a {v)x,y) = {$ s (u)x 2 ,y) = [ ud^ s X2y . 

J T 

Thus V ~ fJ- X2 V , and so n X2y = 0, which implies that ($ s (u)$ a (v)x, y) = 0. 
Therefore 

= $ s (u)$ a (u) = 0 for all u,v £ _4(T). 

Now 


f“(w) = $ a (u)$(v) = $°(u)$°(i;) + $ a (w)$ s (u) = $ a (u)$ a (», 


and so 

<S> s (uv) = &(uv) -<5> a (uv) = ($“(«) +$ s (u)) ($“(u) + $») -$ a (w)$ a (?;) 

= <S> s (u)<f> s (v). 

Therefore, $ a and are bounded representations. 

(ii): It is clear that (P°) 2 = ($°(1)) 2 = $ a (l 2 ) = P°, and similarly (P s ) 2 = 
P s . Moreover, P a P s = $ a (l)$ s (l) = 0 = $ s (l)$ a (l) = P S P° and P° + P s = 
$°(l) + $ s (l) = <F(1) =/. 
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Let C £ {T}' be arbitrary. Then C${u) = <&(w)C for any u £ A(T). Given 
x,y £ H, set ncx,y € M($,Cx,y) and y x ,c* y € M($,x,C*y). Since, for every 
u £ -4(T), we have 

/ udy, C x,y = {${u)Cx,y) = (C$(u)x,y) = ($(u)x,C*y) = / udy XtC * v , 

J T J T 

it follows that y C x, y ~ dx,c* y , and so \x a Cxy ~ d%,c*y and d-Cx, y = ^,c*y 
Then, for every u £ »4(T), 

(<f> a (u)Cx,y) = uddcx, y = ud dx,c* y 

J T J T 

= {<S> a (u)x,C*y) = {C$> a {u)x,y) for all x,y £ H. 

Thus < f>°(i()C' = Ctf> a (u), whence & s (u)C = C$ s (w) follows. Therefore 
$ a (u), $ s (u) £ {T}" for all u £ A(T). 

In particular, P a ,P s £ {T}". 

(iii) : Since P a ,P s £ {T}", it follows that the subspaces H a = P a H and 
7 { s = P S H are hyperinvariant for T. Furthermore, P a P s = p s p a = o and 

P a + ps = I imply that n a +n s = n. 

(iv) : Let us consider the decomposition T = T a +T S , where T a := T\H a and 
T s := T\H S . For any u £ ^4(T), we have $ a (u)P a = P a $ a (u) = <b a (l)<l> a (u) = 
$ a (u), whence <h a (u) = ($ a (u)|77 a )-i-0 follows. The bounded representation 

A(T) -► C(H a ), $g(tt) := <S> a {u)\H a 

satisfies the condition <bg(x) = T a = 3>t“,i(x)> and so = Thus T a is 

an a.c. polynomially bounded operator. Similarly, <F s (w) = 0 + ($ s (u)|'H s ) for 
u £ *4(T), and <1 >tm coincides with the singular representation 

<S> S 0 : A(T) -»• £(H S ), $g(u) := $ s (u)|H s . 

Thus T s is a singular polynomially bounded operator. 

As for uniqueness, let us suppose that the decomposition H = Hi 4- H 2 is 
reducing for T, the restriction T\ = T\H\ is a.c., and T 2 = T\H 2 is singular. 
For every u £ A(TT), we have $(u) = $t,i( u ) = $Ti,i(m) + $t 2 , i( u )- Let us 
consider also the bounded representations $i,$ 2 : A(T) —>• C(H) defined by 
$i(u) := < f> Tljl (u) + 0 and <b 2 (u) := 0 + 3 >t 2 ,i( u )- Since Ti is a.c., it follows 
that $1 is a.c.; similarly, singularity of T 2 implies that <I> 2 is singular. Taking 
into account that $ = $1 + d> 2 , the uniqueness part of Proposition 10 implies 
that $1 = $ a and $ 2 = $ s . Thus Hi = $i(l )H = $ a (l )H = H a and U 2 = 
<5> 2 (l)U = ‘5> s {l)U = U s . 

□ 

Concluding this section we examine the adjoint of a polynomially bounded 
operator T £ C(H). For any function /: T —> C, let /(£) := /(£) (£ e T). 
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For a polynomial p(() = En=o c «C”, we have p(Q = E«=o GiC\ and so 
p(T*) = p(T)*. Since ||p(T*)|| = ||p(T)|| and ||p|| = ||p||, it follows that T* 
is also polynomially bounded, and || < 1 ) t*,o|| = ||$t,o||- Taking uniform limits of 
polynomials, we obtain that for any u £ *4(T), u £ .4(T) and 

u(T*) = u(T)*. 

For any measure p £ M(T), p £ M{ T) is defined by p(w) := p(ZJ), upper 
bar meaning complex conjugation. We shall use the notation M(T,x,y) = 
M($ T ,i,x,y). 

Proposition 14 If T £ CfH) is polynomially bounded, then T* is also polyno¬ 
mially bounded, and 

(i) M(T*,x,y) = {p: p£ M(T,y,x)} for x,y £ U; 

(ii) T is a.c. if and only if T* is a.c.; 

(iii) T is singular if and only if T* is singidar. 


Proof. Given x,y £ H , set p £ M(T,y,x) and v £ M(T*,x,y). For every 
u £ -4(T), we have 


[ udv = (u(T*)x,y) = (u(T)*x,y) = (u(T)y,x) = [ udp 

J T J T 


/ u(C)dp( C)= / udp, 
I T J T 


whence v ~ p follows. Since h £ Hq, for any h £ Hq , we obtain (i). It is easy 
to see that p is a.c. (singular) if and only if p is a.c. (singular, respectively), 
which shows the validity of (ii) and (iii). 

□ 


4 Intertwining relations 

Let T\ £ and T 2 £ CCH 2 ) be polynomially bounded operators, and let 

us assume that Q £ X(T 1 ,T 2 ). Given x £ Hi and y £ H 2 , set p £ M(T 2 , Qx, y) 
and v £ M(T 1 ,x,Q*y). For every u £ »4(T), we have 

/ udp = (u(T 2 )Qx,y) = (u{T 1 )x,Q*y) = / udv, 

J t J T 

hence p ~ v, and so 


M(T 2 ,Qx,y) = M(T 1 ,x,Q*y). 
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Proposition 15 LetT\ £ E(Hi) be an a.c. polynomially bounded operator, and 
let T 2 6 £(% 2 ) be a singular polynomially bounded operator. Then X(Ti,T 2 ) = 
{0} and l(T 2 ,Ti) = {0}. 

Proof. Let Q £ Z(Ti,T 2 ) be arbitrary. Given x £ Hi and y £ H2, and 
setting y £ M{T2,Qx,y) and v £ M(Ti,x,Q*y), we obtain from the previous 
discussion that y ~ v. Since y is singular and v is a.c., it follows that y = 0. 
Then ( Qx,y ) = J T ldy = 0; and since x £ H\,y £ H2 were arbitrary, we 
conclude that Q = 0. Turning to adjoints and applying Proposition 14, we 
obtain that Z(T 2 ,Ti) = {0} also holds. 

□ 

We recall that Ti is a quasiaffine transform of T 2 , in notation: Ti -< T 2 , 
if X(Ti,T 2 ) contains a quasiaffinity, i.e., an injective transformation with dense 
range. 

Proposition 16 Let T\ £ C{Hi) and T 2 £ C(H. 2 ) be polynomially bounded 
operators, and let us assume that Ti -< T 2 . Then 

(i) Tj is a.c. if and only z/T 2 is a.c.; 

(ii) Tj is singular if and only if T 2 is singular. 


Proof. Let us assume that T\ is a.c., and let us consider the Lebesgue de¬ 
composition T 2 = Zf-i-Zf of T 2 . There exists a quasiaffinity Q £ X(Ti,T 2 ). 
Let Pf £ C{H2) be the (oblique) projection onto % 2 . Since the transformation 
Q s £ C{H\,H?f), defined by Q s x := PfQx (x £ Hi), intertwines T\ with Z|, we 
infer by Proposition 15 that Q s = 0. Thus QH\ C H2, and since Q has dense 
range, we obtain that "H 2 = {0}. Therefore T 2 is a.c.. Conversely, assuming 
that T 2 is a.c., the relation Tf -< Tj* yields that Tf is a.c., and then so is Ti too 
by Proposition 14. Statement (ii) can be proved similarly. 

□ 


Let U £ C(K.) be a unitary operator. Relying on the Gelfand transform 
of the abelian C*-algebra generated by U, it can be shown that there exists a 
uniquely determined isometric *-representation $: C(a(U)) —»• C(K.) such that 
<f>(x) = U. Here a(U) C T is the spectrum of U. This $ induces a contractive *- 
representation $: C(T) —>• T(/C), defined by f(U) = $(/) := $(/|(j([/)). Since 
$|P(T) = $c/,o, it follows that U is polynomially bounded with ll^^oll = 1; 
furthermore 4>|„4(T) = 'hyp. It is known also that $ can be represented by 
integration with respect to a uniquely determined spectral measure E: Bj —» 
V(K.)\ see, e.g., [C]. (Here Bj denotes the cr-algebra of Borel sets on T, and 
V{Kf) stands for the set of orthogonal projections on /C.) Namely, for every 
/ € C(T), 

( f(U)x,y)= [ fdE x , y (x,y£lC), 

J T 
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where E X}V G M(T) is the localization of E to x,y, defined by E XiV (lo) := 
(E(ui)x,y) (u) G Bt)- Therefore 

M ( U , x , y) = {E XtV + hdm : h € Hq}. 

We recall that the unitary operator U is called a.c. (singular, resp.), if E x y 
is a.c. (singular, resp.) for every x,y G 1C. The previous relation shows that 
these properties coincide with the corresponding properties considering U as a 
polynomially bounded operator. 

Let us consider the Lebesgue decomposition U = U a 4- U s G L(K = IC a + /C s ). 
Since /C a G Hlat U and U* = U -1 G {[/}', it follows that /C a is reducing for 
U, and so the orthogonal projection Q a G C.{1C) onto /C a commutes with U. 
Therefore Q a \IC s G X(U s ,U a ), whence Q a \IC s = 0 follows by Proposition 15. 
We conclude that K. a is orthogonal to JC S . (For a discussion of Lebesgue decom¬ 
position of unitaries, based on spectral measures, see [Ha].) 

Let us consider now a contraction T G £("H) : ||T|| < 1. It is known that T 
can be decomposed into the orthogonal sum T = T C ®U G C(H = H C ®'H U ) of a 
completely non-unitary contraction T c and a unitary operator U. By Sz.-Nagy’s 
Dilation Theorem T c has a minimal unitary dilation W, acting on a Hilbert space 
/C, containing He- We recall that W is an a.c. unitary operator, T” = P c W n \'H c 
for every n G Z + , and K, = V^L 1 W~ n 'H c . (Here P c G £(/C) is the orthogonal 
projection onto 1~L C .) In connection with the theory of contractions we refer to 
[NFBK]. For every p G V(T), we have 

\\p(T c )\\ = \\P c p(W)\H c \\ < \\p(W)\\ < \\p\\, 

what is called the von Neumann inequality , and so T c is polynomially bounded. 
Since, for any x,y G He, (u(T c )x,y) = (P c u(W)x,y) = ( u(W)x,y) holds for 
every u G ^4(T), we infer that 


M(T C , x, y) = M(W, x, y). 

Therefore T c is a.c.. Considering the decomposition T = T c © U a © U s , we can 
see that the a.c. component of T is T° = T c © U a , while the singular part is 

T s = us. 

If T G C(T~L) is a polynomially bounded operator, U G £(/C) is a singular 
unitary operator, and T -< U, then T is singular by Proposition 16. The fol¬ 
lowing theorem, which in different forms can be found in [AT] and [M4], states 
that there must be a much stronger relation between T and U in that case. 

Theorem 17 If T G C(H) is a singular polynomially bounded operator, then 

(i) the funcional calculus $t,i : A(T) —>• C(fH) can be extended to a bounded 
representation : C(T) —> C(H); 

(ii) the operator X G X(T, U) is invertible, whenever ( X , U) is a unitary 
asymptote ofT; and so T is similar to the singular unitary operator U. 
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Proof, (i): We repeat the procedure carried out in the first part of the proof 
of Proposition 10. We shall write $ = $^1 and $ = i for short. For any 
x,y £ H, there exists a unique singular measure y XtV in M(T,x,y). Let us 
consider the bounded sesquilinear mapping 

w T M( T), w T (x, y) = y, XtV . 

For any g £ C(T), A g : M( T) — > C is defined by A g (y) = J T gdy. The bounded 
sesquilinear functional 


wr,g '■= A g o wt : H x H —> C 


uniquely determines an operator <£(< 7 ) £ C(H): 


{®(g)x,y} = w T , g {x,y) 



(x, y £ %). 


We know that ||$(< 7 )|| < || < f > ||||g||- It is clear also that <F is linear. We have to 
show yet that <E> is multiplicative. 

For v £ -4(T), we have v dy x y = \x Xv y by Lemma 12. Thus, for any / £ 

cm 


($(fv)x,y} = f fvdg x , y = [ f dfi Xv , y = (®(f)x v ,y) = ($(f)${v)x,y), 

J T J T 

whence &(fv) = follows. Setting yj = for every v G A(T), 

f vf dn xy = ($(fv)x,y) = {$(f)<Z>(v)x,y} = ($(v)x,y* f ) = f vd/j, x , v *. 

J T J T f 

Thus, the singular measures / dn x , y and y x ,y* are analytically equivalent, and 
so they must coincide: f dy XtV = y x , y y Taking any g £ C(T), 

(' ®(fg)x,y} = f fgdn x , y = [ gdy x . y . = ($(g)x,y* f ) = ($(f)$(g)x,y) 

J T J T 

holds for every x,y GH. Consequently, $(fg) = $(/)$(g). 

(ii): By the multiplicativity of $ we infer that T is invertible and c E > (x™) = T n 
holds, for every n £ Z. Since ||T , "|| < ||$|| = ||$|| for all n £ Z, a well- 
known theorem of Sz.-Nagy yields that T is similar to a unitary operator V ; 
see [N]. Let Q £ I(T,V) be invertible. The polynomially bounded operator T 
is necessarily power bounded, and so it has a unitary asymptote (X, U). There 
exists a unique Y £ T(U, V ) such that Q = YX. Since Q is invertible, it follows 
that X is bounded from below. The isometry U\X is singular by Proposition 16. 
Considering its Wold decomposition, we can see that U\X is unitary. Hence XH 
is reducing for U, and so the minimality of ( X , U ) yields that X is a surjection. 
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Therefore X is invertible, and then Y must be invertible too. Consequently, the 
operators T, U and V are similar to each other. 

□ 

We conclude this section with a discussion of further intertwining relations. 
Since Hilbert space contractions have a rich theory (see [NFBK]), it would be im¬ 
portant to know how more general operators can be related to contractions. An¬ 
swering a question posed by Sz.-Nagy, Foguel gave examples for power bounded 
operators which are not similar to contractions; see [F]. Pisier answered nega¬ 
tively also a more delicate question of Halmos, showing that not every polynomi- 
ally bounded operator is similar to a contraction; see [Pi]. Muller and Tomilov 
proved, giving negative answer for a question of the author posed in [Kl], that 
there are power bounded operators of class C\\ which are not similar to con¬ 
tractions; see [MT], We recall that T € C\\ means Hq(T) = Ho(T*) = {0}, 
and in that case T is quasisimilar to its unitary asymptote U : T ~ U, that is 
T -<U and U -<T hold simultaneously. 

Question 18 If the polynomially bounded operator T is of class C\ \, is T sim¬ 
ilar to a contraction? 

For additional conditions under which a power bounded operator is similar 
to a contraction see [G2]. We note yet that by a fundamental characterization 
due to Paulsen, exactly those operators are similar to contractions which are 
completely bounded; see [Pa]. 

Quasisimilarity is a much weaker relation than similarity, but it preserves 
also numerous properties and plays important role in the classification of opera¬ 
tors; see, e.g., [B] and [DH]. Muller and Tomilov showed that a power bounded 
operator T is not necessarily quasisimilar to a contraction [MT]. On the other 
hand, Bercovici and Prunaru proved that if T is a polynomially bounded oper¬ 
ator, then there exist contractions Ti and T 2 such that T\ -< T -< T 2 ; see [BP], 
As far as we know, the following questions are still open. 

Question 19 Is every polynomially bounded operator quasisimilar to a contrac¬ 
tion? 

Question 20 If the power bounded operator T is quasisimilar to a singular 
unitary operator V, does it follow that T is similar to V ? 

The latter question is connected with Theorem 17, and was posed in [Kl]. 
Partial answer for it can be found in [Gl]. 


5 i7°°-functional calculus 

The Hardy class H°° is the weak-* closed subalgebra of L°°(T), consisting of 
those functions / whose Fourier coefficients satisfy the condition /(—n) = 0 
for n € N. H°° can be identified as the dual of the Banach space Ir( T)/Hq. 
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For a detailed study of this class see [Ho]. Sz.-Nagy and Foias introduced H°°- 
functional calculus for a.c. contractions, and thoroughly exploited its properties 
in their theory of contractions; see [NFBK] and [B], In Chapter 2 of [CP] H°°- 
functional calculus is defined for a polynomially bounded operator T acting on 
a complex Banach space X, which is stable, that is limn-**, ||T"a:|| = 0 holds 
for avery x G X. In [Ml] and [M3] Mlak considered representations of general 
function algebras. Here we follow Mlak’s method of elementary measures, con¬ 
centrating on H°°, and providing detailed study of the calculus in this case. 
We note that in [K 6 ] our approach was based on the unitary asymptote in the 
Ci.-case, that is when 'Hq{T) = {0}. 

We recall that C(T~L) is the dual of the Banach space Ci(H) of trace class 
operators. Namely, the mapping A: C(7i) —> Ci(%) # , defined by [A, A(C')] = 
tr(AC), is a Banach space isomorphism; see [Sch]. 

We say that the operator T G C(fH) admits an H°°-functional calculus, if 
there exists a weak-* continuous representation <Ft, 2 : H°° —> £(%) such that 
H*t, 2 (x) = T. In that case we use the notation <Ft, 2 (/) = /(T) (/ G H°°). In 
the following proposition we collect some basic facts about this calculus. 

Proposition 21 If T G C(H) admits an H°°-functional calculus, then 

(i) 3 >t ,2 is uniquely determined; 

(ii) $ 7^2 is norm-continuous; 

(iii) T is polynomially bounded, <Ft,2|A(T) = <Ft,i, and ||<Ft,2|| = ||$t,i||; 

(iv) H°°(T) := ran $ T , 2 C {T}"; 

(v) LatT = Lat H°°(T); 

(vi) for every Ai G LatT, T\M admits an H°°-functional calculus, and 
f(T\M) = f(T)\M (/ G H°°). 

Proof, (i): Let us assume that '!>,: H°° —> C(fH) is a weak-* continuous repre¬ 
sentation, for j = 1,2, and that $i(y) = , I > 2 (x) holds. Then ^(p) = $2 (p) is 
true, for every polynomial p G T(T). Taking an arbitrary / G H°°, the Cesaro 
means 

= E (‘ - syr) 

fe =0 v 7 

of the Fourier series of / converge to / in the weak-* topology, as n —> 00 ; see, 
e.g., Chapter 2 in [Ho]. By continuity we obtain that $1 (/) = d> 2 (/). 

(ii): For short we write $ = 'Ft, 2 - The unit ball (H°°) 1 := {/ G H°° : 
ll/ll < 1} is weak-* compact by the Banach-Alaoglu Theorem. Since $ is 
weak-* continuous, it follows that $(U°°)i is weak-* compact in C{H). Thus 
{[A, $(/)] : / G (H°°) 1 } is compact and so bounded in C, for every A G 
Then the Uniform Boundedness Principle yields that &(H°°)i is bounded in 
£("H), which means the norm-continuity of <F. 
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(iii): Since d>|P(T) = d>T,o> we infer from (ii) that $t,o is bounded: ||$t,o|| < 
||$||. Hence T is polynonrially bounded, and $|„4(T) = $t,i- Furthermore, for 
any / £ H°°, we have || c E > (o'/,, l )|| < ||d> 7 ;o|| • ||/|| (n £ N). Taking into account 
that u>*-lim„ <£( 07 ,„) = d>(/), we conlude that ||$(/)|| < ||d>T,o|| • ||/||- Therefore 
ll^ll = ||®T,o|| = ||$T,l||- 

The statements (iv), (v), (vi) can be easily derived from the fact that H°° 
is the (sequentially) weak-* closure of 'P(T). 

□ 

The following lemma shows that it is enough to check seemingly weaker 
conditions in order to prove that a mapping is an i?°°-functional calculus. 

Lemma 22 If <f>: H°° —>• C(TL) is a linear mapping, $|^4(T) is a representa¬ 
tion, and lim„ ($(/„):£, y) = 0 for every x, y £ TL whenever w*-lim„ f n = 0, then 
$ is an H°°-functional calculus. 

Proof. Let us assume that the sequence {f n }ffL i hr H°° converges to zero in the 
weak-* topology. By the assumption, the operators {$(/„)}))T 1 converge to zero 
in the weak operator topology (wot). The Uniform Boundedness Principle yields 
that M = sup{||d>(/„)|| : n G N} < oo. Taking an arbitrary operator A G C\(fH), 
let us consider the Hilbert-Schnridt decomposition A = s k%k <8> Vk- Here 
{xk}k and {yk}k form orthonormal systems, Sfc > 0 for all k, and Y2k s fc = 
PHi < oo. These conditions imply that 

lim tr(d>(/„)A) = lirn V' s k ($(f n )x k , Vk) = 0. 

n—>• oo n—too L —' 

k 

Thus the functional o $ is sequentially weak-* continuous, where the weak-* 
continuous functional A^: E(H) —>■ C is defined by Aa(C) = tr(CA). By the 
Krein-Smulian Theorem A^ o $ is weak-* continuous; see Corollary V.12.8 in 
[C]. Therefore, d> is also weak-* continuous; see Proposition 1.3.2 in [KR]. 

Setting any f,g G H°°, we know that ®{(Tf, n (Tg,k) = holds 

for every n, k G N. Since w*-lim„ cr/ >n = /, we infer by the sequentially weak- 
*-wot continuity of d> that $(/cr Si fc) = d>(/)d>(cr Sj fc). Tending now k to infinity, 
we conclude that d>(/<?) = d>(/)d>(c/). 

□ 

Theorem 23 The operator T G C(Tl) admits an H°°-functional calculus, if 
and only if T is an absolutely continuous polynomially bounded operator. 

Proof. Let us assume that T admits an if°°-functional calculus. Then T is 
necessarily polynomially bounded by Proposition 21. Suppose that T is not a.c., 
and consider the Lebesgue decomposition T = T a + T s G C(H = H a +'H S ), 
where H s ^ {0}. In view of Proposition 21, T s admits an ff°°-functional 
calculus. We know by Theorem 17 that T s is similar to a singular unitary 

operator U G £(/C), let Q G I(T,U) be invertible. It is clear that d>[/ j2 (/) := 

Qd> Ti2 (/)Q' 1 (/ G H°°) defines an i7°°-calculus for U. Let E denote the 

spectral measure of U. Taking a non-zero vector x G /C, let us consider the 
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positive measure E x>x . We know that E x>x is singular and E x<x ( T) = ||cc|| 2 > 0. 
In view of regularity we can find a compact set K C T such that m{K) = 0 and 
E x ,x(K) >0- By a result of Rudin there exists a function u £ A(T) such that 
u(() = 1 for all £ £ K , and \u(()\ < 1 for all £ £ T \ K\ see page 81 in [Ho]. 
Since u>*-lim„ u n = 0, regarding the continuity properties of <&u,2 it follows that 
\im. n {u n (U)x, x) =0. However, 


lim ( u n (U)x , x) = lim / u n dE x x = E x X (K) > 0. 

n—>oo n—>oo J 

We arrived at a contradiction, and so the polynomially bounded operator T 
must be absolutely continuous. 

Let us assume now that T is an a.c. polynomially bounded operator. Let 
/ £ H°°. Given any x,y £ 7f, let us choose an elementary measure p x , y £ 
M(T,x,y). Being abolutely continuous, /j, XtV = g X:V dm where g XtV £ L 1 (T); 
hence the integral J T f dji X) y = J T fg XjV dm can be formed. Any other v £ 
M(T,x,y) is of the form v = g x<y + hdm with h £ Hq, and so f T fdv = 
J T fdg Xi y + J T fhdm = jjfdux^y, because fh £ Hq. Let us consider the 
well-defined mapping 

wf.'Hx'H—t C, Wf(x,y) = 

It is easy to check that Wf is sesquilinear, and ||ic/|| < ||/|| ■ || < 1 ) t,i||- There exists 
a unique operator $(/) £ C(%) such that 

mf)x,y) = w f (x,y) = / fdg x , y (x,y £%). 

JT 

It is clear that $: H°° —> C(H) is linear, and <b|A(T) = 4 >t,i is a representation. 
If {/raj^Li is a sequence in H°° converging to 0 in the weak-* topology, then 



lim (®{f n )x,y) = lim / f n gx, y dm = 0 (x,y£'H). 

n —>oo n —>oo 

Applying Lennna 22 we obtain that $ is an If °°-functional calculus for T. 

□ 


Proposition 24 If T £ C(fH) admits an H°° -functional calculus, then so does 
its adjoint T*. Furthermore, f(T*) = f(T)* holds for every f £ H°°. 

Proof. We have seen that T is an a.c. polynomially bounded operator if and 
only if so is T*; see Proposition 14. Furthermore, for any polynomial p £ V(T) 
we have p(T*) = p{T)*. Let / € ff°° be arbitrary. Since w*-lim„ = f 
and w*-lim n cr/ in = /, it follows that converges to /(T*), and 

{<7 f,n(T)} n converges to f(T) in the weak operator topology. The latter condi¬ 
tion implies that {df, n (T)*} n converges to f(T)* in wot. Therefore, the equal¬ 
ities ct/,„(T*) = af >n (T)* (n £ N) yield that f(T)* = f(T)*. 

□ 
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6 Quasianalytic operators 

Let T £ £{%) be an a.c. polynomially bounded operator. Let ( X, U) be a 
unitary asymptote of T, where U £ £(JC). Taking the Lebesgue decomposition 
U = U a © U s £ £()C — /C° ® K, s ), and considering the transformation Y £ 
T(T,U S ), defined by Yh = P s Xh, we infer by Proposition 15 that Y = 0. 
Hence XTL C Kf, and it follows by the minimality of (X,U) that X s = {0}. 
Therefore, U is an a.c. unitary operator. 

Let E: £>t —> V(X) denote the spectral measure of U. For any x, y £ H, the 
localization of E at Xx, Xy is of the form Ex x ,Xy = Wx,y dm, where w XtV £ 
L 1 ( T) is the asymptotic density function of T at x,y. We note that 

Exx,x y £ M(T, x, Ay) 

with A = X*X (see the beginning of Section 4), and Ex x ,Xx is the unique 
positive measure in M(T,x,Ax). 

The measurable set 


u(T,x) := {C £ T : w x>x {Q > 0} 

is called the local residual set of T at x. Considering the functional model of 
the unitary operator U, we may easily check that the hyperinvariant subspace 
of U generated by Xx is just the spectral subspace corresponding to u>(T,x ): 

E(w(T, x))K = W{FXx : F £ {[/}'} £ Hlat U. 

More precisely, u(T, x) is the smallest measurable set on T such that E(w{T, x))X 
contains Xx, that is Xx £ E{a)K. implies m(co(T,x) \ a) = 0; see, e.g., [C]. 

Proposition 25 The local residual setiv(T,x) is independent of the particular 
choice of the unitary asymptote (X, U). 

Proof. Let (X',U r ) be another unitary asymptote of T; E’ is the spectral 
measure of U' £ £(JC'), and uj'{T,x) is the local residual set of T at x, defined 
via E'. We know that there is an invertible transformation Z £ I (U, U r ) such 
that X' = ZX. Let us consider the polar decomposition Z = W\Z\, where 
\Z\ > 0 is invertible and W is unitary. It is easy to verify that \Z\ £ {U}' and 
WU = U'W. Since the vector X'x = ZXx belongs to the subspace 

ZE{u{T,x))K = W\Z\E(uj(T,x))IC = WE(uj{T,x))\Z\K, = E'{w{T, x))K!, 

it follows that m(u'(T,x) \ oj(T,x)) = 0. Changing the roles of (X,U) and 
(X',lf) we obtain that the symmetric difference uj(T, x) A uj' (T, x) is of zero 
Lebesgue measure, and so they can be considered identical. Note that the 
Radon-Nikodym derivative w X:X , and so ui(T,x) also are determined up to sets 
of zero Lebesgue measure. 

□ 
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Let us consider the functional calculus $t, 2 : H°° -5- C{H), f f(T) for T. 

a 

We write Kt = ||$t, 2 || for short. We recall that for f,g £ H°°, f -< g means 
that \f(z)\ < \g(z)\ for all z £ D := {C € C : |£| < 1}- Then / = gh, where 
h £ H°° and \\h\\ < 1. Thus ||/(T)a;|| = \\h,(T)g(T)x\\ < K T \\g(T)x\\ ( x £ H), 

a 

that is f(T) -< KTg(T). (If T is a contraction, then I\t = 1 and so <1 >t, 2 is 

a 

monotone.) Let F = {/ n }£Li be a decreasing sequence in H°°, i.e. f n +i -< fn 
for every n. The measurable limit function <Pf(Q = lim„ |/ n (C)| is defined for 
almost every ( £ T. Set 


N f :={ C e T : g? F (0 > 0}. 

We say that F is asymptotically non-vanishing on the measurable set a C T, if 
m(a n Np) > 0. 

Proposition 26 If inf{||/ n (T)a;|| : n £ N} = 0, then liiUn^oo ||/„(T)s|| = 0. 
Furthermore, 

Ho(T,F) := {x£H: lim \\f n (T)x\\ = 0} 

n—> 00 

is a hyperinvariant subspace ofT. 


a 

Proof. The first statement follows from the condition /„+i (T) -< Krf n {T) (n £ 
N), while the second one is a consequence of sup{||/ n (T)|| : n £ N} < 00 . 

□ 

The following theorem shows that the local residual set is responsible for 
local stability. 

Theorem 27 (a) If x £ Wq(T, F), then m(Nf fl u(T, x)) = 0. 

(b) If m{Np flo i(T,x)) = 0, then there exists a strictly increasing mapping 
t: N — > N such that x £ Ho(T,G), where G = {x T ^fn}%Li- 

Proof. For any x £ H, we have 

\\Xf n (T)x\\ 2 = \\fn{U)Xx\\ 2 = f \f n \ 2 w X}X dm (n£ N), 

J T 

and the latter integral converges to f T tp 2 F w x , x dm by Lebesgue’s Dominated 
Convergence Theorem, as n —> 00 . Hence (a) follows, and we obtain (b) also by 
applying Theorem 5. Note that pc = Pf- 

□ 

The previous theorem tells us that if the decreasing sequence F = {f n }'^Li 
in H°° is asymptotically non-vanishing on u(T,x), then x ^ Ho(T,F), that is 
the vector-sequence {f n (T)x}?f =1 is asymptotically non-vanishing. On the other 
hand, if F is asymptotically vanishing on w(T, x), then a modified (strengthened) 
sequence {T' r ^"^/„(T)a;}))T 1 is asymptotically vanishing (stable) with a suitable 
mapping r. 
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It is known that the measurable sets on T form a complete lattice, if we 
disregard of sets of measure zero. Hence, for any measurable sets a , /? on T, 
we shall write aCft a = /3 or a ^ /?, when m(a \ /3) = 0, m(a A ft) = 0 or 
m(ai A /3) >0, respectively. The following statement can be proved using the 
Gelfand transform of the abelian Banach algebra L°°(T); see Section 11.13 in 
[R]. Here we sketch an elementary proof. 

Lemma 28 For any system of measurable sets on T, there exist a 

smallest measurable set To = Wi^icoi containing all coi, and a largest measur¬ 
able set co = Ajg/Wj contained in all coi. Furthermore, To and to are uniquely 
determined. 

Proof. Let O be the system of those measurable sets, which contain (in the 
extended sense) every w,; (* £ I). It is clear that is closed under countable 
intersection. Set a = mi{m(co) : co € ST}, and select a sequence {c o n }^ =1 in 
Q so that lin i n m(co n ) = a. Then it is easy to verify that the measurable set 
To = f\ff =1 co n has the required properties. The statement about co can be proved 
similarly, considering the system Q of those measurable sets, which are contained 
in every coi (i £ I), and then taking b = sup{m(w) : co £ O}. 

□ 

The set 

co(T) := W{co(T,x) : x £ %} 

is called the residual set of T. It is clear that co(T) is the measurable support of 
the spectral measure E of U, that is E(a) = 0 exactly when m(a fl co(T)) = 0. 
Furthermore, co(T) = 0 means that /C = {0}, that is 'Hq(T) = H, in which case 
T is called of class Co.. 

The set 

7r(T) := A {co(T,x) : 0 ^ x £ U} 

is called the quasianalytic spectral set of T. In view of Theorem 27, this is 
the largest measurable set with the property that 7io(T,F) = {0}, whenever 
F = {f n \n :Li is a decreasing sequence in H°°, asymptotically non-vanishing on 
7r(T). We can characterize 7r(T) also with more general seciuences, as in the next 
proposition. (For related results in connection with test sequences of stability, 
see [KSzl].) 

Proposition 29 If F = {f n }f?= i Is a bounded sequence in Ft 00 such that 
limsup„ ||x 7 t(t) fn II 2 > 0, then the hyperinvariant subspace 

H 0 (T, F) := {x £ U : lim ||/„(7>|| = 0} 

n—>oo 

reduces to zero. Furthermore, 7r(T) is the largest measurable set having this 
property. 

Proof. Let F = {f n }%L i be a bounded seciuence in H°° such that c = 
limsup ra f n r T \\fn \ 2 dm > 0. Set M = sup{||/„||oo : n £ N} < oo. Further¬ 
more, given 0 jT x £ H, let a(x,N) := {£ £ 7r(T) : w XlX (Q > IV -1 } for N £ N. 
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We can select N so that M 2 m(Tr(T)\a(x, N)) < c/2. For every n £ N, we have 



/ \fn\ 2 w x ,xdm > N 1 / \f n \' 2 dm 

h Ja(x,N) 


<T) 


\f n \ 2 dm — [ \f n \ 2 dm 

J-k(T)\cx.{x,N) J 

> N - 1 {\\x AT )fn .111 - M 2 m(n(T) \ a(x, N))) 


Hence 


limsup ||X/„(T)a-|| 2 = limsup / \f n \ w x , x dm>N c/2 > 0, 

n—> oo n—too Jj 

and so x is not stable for F. Therefore 'H 0 (T,F ) = {0}. 

Let j3 be a measurable set on T, larger than n(T) : /3 D 7t(T) and /? / 7 t(T). 
By the definition of 7r(T) we can find a non-zero xo £ T~L so that /3 is not contained 
in u(T,x o), that is A = 0 \uj(T,xq) is of positive measure. Let fo £ H°° be 
an outer function satisfying the condition |/o| = xa + (1/2)xt\Ai an d let us 
consider the decreasing sequence F 0 = {f//}//Li in H°°, with ipp 0 = xa- In 


view of Theorem 27, there exists a strictly increasing r: N 
x 0 £ 7i 0 (T,F) with F = {x T ^/o }5£Li- It is clear also that 


N such that 


lim llx^/oll = m(A) > 0, 


which completes the proof. 

□ 

The absolutely continuous polynomially bounded operator T £ C(H) is 
quasianalytic, if 7r(T) = w(T) / 0. We note that ir(T) C oj{T) is evident 
from the definition. Theorem 3 in [KSz2] shows that our definition is consistent 
with the one given in the contractive case. 

The following theorem illuminates the importance of quasianalytic operators, 
showing that the challenging Hyperinvariant Subspace Problem can be reduced 
to this class in the asymptotically non-vanishing case. 


Theorem 30 Let T £ C(H) be an absolutely continuous polynomially bounded 
operator, which is asymptotically non-vanishing: Ho(T) / H. If T is not quasi¬ 
analytic, then it has a non-trivial hyperinvariant subspace. 


Proof. By our assumption we can find non-zero vectors Xi,X 2 £ H so that 
lo(T,x 2 ) is not contained in lj(T,x 1 ), that is the set A = ui(T,x 2 ) \ u>(T, x{) is 
of positive measure. There exists a decreasing sequence F = {f n }%Li in H°° 
such that x± £ Hq{T,F) and Np = A; see the second part of the proof of 
Proposition 29. The latter relation implies by Theorem 27 that x -2 / HoiT, F). 
Therefore, H.o(T, F ) is a proper hyperinvariant subspace of T. 

□ 


The preceding proof immediately yields the following statement. 
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Proposition 31 LetTi £ F(Hi) andT 2 £ X(H 2 ) be a.c. polynomially bounded 
operators. If ui(T 2 ,X 2 ) is not contained in lo(Ti,x\), then there exists a decreas¬ 
ing sequence F in H°° such that x\ £ Ho(Ti,F) and X 2 fL Hq(T 2 , F). 

We proceed with some intertwining relations. 

Proposition 32 Let T\ £ C(H 1 ) and T 2 £ C(H 2 ) be a.c. polynomially bounded 
operators. 

(a) If Y £l{Ti,T 2 ), then oj(Ti,x) D uj(T 2 ,Yx) holds for every x £ Hi. 

(b) If I(Xj,T 2 ) contains an injection Y, then 7r(Xj) D ^(X^). 

(c) If both Z(Tj,T 2 ) and Z(T 2 ,Tj) contain injections, then 7r(Tj) = 7r(T2). 

(d) If Z(Tj,T 2 ) contains a transformation Z with dense range, then w(Tj) 2> 

w(T 2 ). 

(e) If Ti ~ T 2 , then 7r(Tj) = ^(T^) and w(Tj) = u;^)- 

(f) If Ti ~ T 2 and T\ is quasianalytic, then X 2 is also quasianalytic. 

Proof. The equations Yf n {T\) = f n (T 2 )Y (n £ N) show that YHq(T\,F) C 
Hq{T 2 ,F) holds, for any decreasing sequence F = {f n }%Li hr H°°. In view of 
Proposition 31 we infer that u>(Ti,x) D lo(T 2 ,Yx) for every x £ Hi. Assuming 
that Y £ X(Ti,T 2 ) is injective, this relation yields 

7t(T 2 ) = A {u(T 2 ,y) : 0 ^y £ H 2 } C A{w(Ti,x) : 0 ± x £ Hi} = 7t(Ti). 

Let us assume now that Z £ T(Ti, T 2 ) has dense range. Let (Xj,Uj) be a 
unitary asymptote of Tj (j = 1,2). Since X 2 Z £ X(Ti,U 2 ), there exists a 
unique Z £ X(Ui, U 2 ) such that X 2 Z = ZXi. It follows that 

(ZK.i)~ = (ZV nGN Ui n XiHi)~ = V nm Ui n ZXiHi = V n&N U^ n X 2 H 2 = /C 2 . 

Considering the polar decomposition Z = W\Z\ we obtain that W £ X(Ui, U 2 ) 
is a coisometry, and so U 2 is unitarily equivalent to the restriction of U\ to its 
reducing subspace (ker W)- 1 . Therefore, the measurable support w(Xj) of the 
spectral measure of U\ contains the measurable support w(T 2 ) of the spectral 
measure of t/ 2 . 

□ 


Quasianalicity determines the asymptotic behaviour of the operator. 

Proposition 33 If the a.c. polynomially bounded operator T £ C(H) is quasi¬ 
analytic, then T is of class C 10 : Hq(T) = {0} and Ho(T*) = H. 
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Proof. Taking the decreasing sequence Fq = i n H°°, we infer that 

Ha (T) = Hq{T,F 0 ) = {0} since Np 0 H 7r(T) = 7r(T) ^ 0; see Proposition 29. 
Assuming that Ho(T*) ^ H, let us consider a unitary asymptote ( X*,U *) of 
the adjoint T*. Then the a.c. unitary operator U * acts on a non-zero space /C*, 
and X* G Iff/*,T) is a non-zero transformation. The equations X*f7* = TX* 
and X* = TX*f7* imply that the subspace kerX* is reducing for Cl*, because of 
kerl C Ho{T) = {0}. Let us consider the restriction f/* ;0 of [/* to the non-zero 
reducing subspace /C» 0 kerX*. Since IfI) contains an injection, we infer 
by Proposition 32 that 7r(f/* >0 ) D 7r(T). However, it is clear that 7r(C/*,o) = 0 
holds for the a.c. unitary operator U* t o, and so 7r(T) = 0, what is a contradiction. 

□ 

Now we examine the effect of taking orthogonal sums. 

Proposition 34 Let Tj G £( Hj) be a.c. polynomially bounded operator acting 
on non-zero space, for j = 1, 2, and let us form T = T\ ©T 2 G C(H = H\ ®H 2 ). 

(a) Then T is also an a.c. polynomially bounded operator with 

7r(T) = 7r(Ii) fi 77 ( 12 ) and cj{T) = w(Tj) U u(T 2 ). 

(b) T is quasianalytic if and only i/Tj and T 2 are quasianalytic and 7r(Ii) = 

*(r 2 ). 

Proof. Setting a: = aq © x 2 , y = yi © yi G H, and pn G M[T\,xi,y\), y,2 G 
M (I 2 , X 2 , 2 / 2 ), we have 

(u(T)x, y) = (u(Ti)xi,yi) + (u(T 2 )x 2 ,y2} = / ud(yi + p 2 ) 

J T 

for every u G *4(T); hence /ri + y 2 G AI(T,x,y). Therefore T is an a.c. polyno¬ 
mially bounded operator. 

Let ( Xj,Uj ) be a unitary asymptote of Tj, for j = 1, 2. We know that (X, U) 
will be a unitary asymptote of T, where X = X\ © X 2 and U = Ui © U 2 ; see 
Corollary 6. Furthermore, if E is the spectral measure of U, then E\K,j will be 
the spectral measure of Uj (j = 1,2). Thus, for any x = X\ © x 2 G H we have 
Exx,Xx = E Xl x u x 1 xi +Ex 2 X 2,x 2 x 2 , whence u(T,x) = u(T u x x ) Uw(T 2 , x 2 ) can 
be derived. From here the statement follows. 

□ 


We conclude with the properties of invariant subspaces. 

Proposition 35 Let T G C{H) be an a.c. polynomially bounded operator, and 
let us assume that A4 G Lat T is non-zero. 

(a) Then T\A4 is also an a.c polynomially bounded operator with 

7 r(T) C tt(T|A4) C u{T\M) C u(T). 

(b) If T is quasianalytic, then T\J4 is also quasianalytic and tt(T\M) = 7 r(I). 
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Proof. It is clear that M(T\M.,x,y) = M(T,x,y) holds for every x,y £ M. 
Hence, T\M. is an a.c. polynomially bounded operator. We know that if (X, U) 
is a unitary asymptote of T, then (X\M, XJ\M) is a unitary asymptote of T\M; 
see Corollary 7. Thus, uj(T\M,x) = lu(T,x) holds, for every x £ M. 

□ 

We recall that if T 6 £(77) is an a.c. contraction with oj(T) = T, then 
there exists Ai £ LatT such that T|A4 is similar to the simple unilateral shift 
S £ C(H 2 ), Sf = xf; even more, these shift-type invariant subspaces span the 
whole space 77 (see Theorem IX.3.6 in [NFBK] and [K7]). 

Question 36 Let T £ £(77) be an a.c. polynomially bounded operator with 
w(T) = T (or ir(T) = T). Does there exist a non-zero Ai £ LatT such that 
T\M is similar to a contraction? 

If 7r(T) = T, then such an Ai clearly contains a subspace M' £ Lat T, where 
T \Ai' is similar to S. It would be interesting to know whether Pisier’s construc¬ 
tion provides completely non-contractive (i.e., having no A4 £ LatT \ {0} 
such that T\M is similar to a contraction) absolutely continuous polynomially 
bounded operators in the case, when w(T) = T or 7r(T) = T. 
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